Tapping into non-English-language science for the conservation of global biodiversity.

The widely held assumption that any important scientific information would be available in English underlies the underuse of non-English-language science across disciplines. However, non-English-language science is expected to bring unique and valuable scientific information, especially in disciplines where the evidence is patchy, and for emergent issues where synthesising available evidence is an urgent challenge. Yet such contribution of non-English-language science to scientific communities and the application of science is rarely quantified. Here, we show that non-English-language studies provide crucial evidence for informing global biodiversity conservation. By screening 419,679 peer-reviewed papers in 16 languages, we identified 1,234 non-English-language studies providing evidence on the effectiveness of biodiversity conservation interventions, compared to 4,412 English-language studies identified with the same criteria. Relevant non-English-language studies are being published at an increasing rate in 6 out of the 12 languages where there were a sufficient number of relevant studies. Incorporating non-English-language studies can expand the geographical coverage (i.e., the number of 2° × 2° grid cells with relevant studies) of English-language evidence by 12% to 25%, especially in biodiverse regions, and taxonomic coverage (i.e., the number of species covered by the relevant studies) by 5% to 32%, although they do tend to be based on less robust study designs. Our results show that synthesising non-English-language studies is key to overcoming the widespread lack of local, context-dependent evidence and facilitating evidence-based conservation globally. We urge wider disciplines to rigorously reassess the untapped potential of non-English-language science in informing decisions to address other global challenges. Please see the Supporting information files for Alternative Language Abstracts.

The CEEDER database of evidence reviews: An open-access evidence service for researchers and decision-makers.

Evidence-informed decision-making aims to deliver effective actions informed by the best available evidence. Given the large quantity of primary literature, and time constraints faced by policy-makers and practitioners, well-conducted evidence reviews can provide a valuable resource to support decision-making. However, previous research suggests that some evidence reviews may not be sufficiently reliable to inform decisions in the environmental sector due to low standards of conduct and reporting. While some evidence reviews are of high reliability, there is currently no way for policy-makers and practitioners to quickly and easily find them among the many lower reliability ones. Alongside this lack of transparency, there is little incentive or support for review authors, editors and peer-reviewers to improve reliability. To address these issues, we introduce a new online, freely available and first-of-its-kind evidence service: the Collaboration for Environmental Evidence Database of Evidence Reviews (CEEDER: www.environmentalevidence.org/ceeder). CEEDER aims to transform communication of evidence review reliability to researchers, policy-makers and practitioners through independent assessment of key aspects of the conduct, reporting and data limitations of available evidence reviews claiming to assess environmental impacts or the effectiveness of interventions relevant to policy and practice. At the same time, CEEDER will provide support to improve the standards of future evidence reviews and support evidence translation and knowledge mobilisation to help inform environmental decision-making.

Assessing the risk of bias in choice of search sources for environmental meta-analyses.

Results of meta-analyses are potentially valuable for informing environmental policy and practice decisions. However, selective sampling of primary studies through searches exclusively using widely used bibliographic platform(s) could bias estimates of effect sizes. Such search strategies are common in environmental evidence reviews, and if risk of bias can be detected, this would provide the first empirical evidence that comprehensiveness of searches needs to be improved. We compare the impact of using single and multiple bibliographic platform(s) searches vs more comprehensive searches on estimates of mean effect sizes. We used 137 published meta-analyses, based on multiple source searches, analyzing 9388 studies: 8095 sourced from commercially published articles; and 1293 from grey literature and unpublished data. Single-platform and multiple-platform searches missed studies in 100 and 80 of the meta-analyses, respectively: 52 and 46 meta-analyses provided larger-effect estimates; 32 and 28 meta-analyses provided smaller-effect estimates; eight and four meta-analyses provided opposite direction of estimates; and two each were unable to estimate effects due to missing all studies. Further, we found significant positive log-linear relationships between proportions of studies missed and the deviations of mean effect sizes, suggesting that as the number of studies missed increases, deviation of mean effect size is likely to expand. We also found significant differences in mean effect sizes between indexed and non-indexed studies for 35% of meta-analyses, indicating high risk of bias when the searches were restricted. We conclude that the restricted searches are likely to lead to unrepresentative samples of studies and biased estimates of true effects.

Ignoring non-English-language studies may bias ecological meta-analyses.

Meta-analysis plays a crucial role in syntheses of quantitative evidence in ecology and biodiversity conservation. The reliability of estimates in meta-analyses strongly depends on unbiased sampling of primary studies. Although earlier studies have explored potential biases in ecological meta-analyses, biases in reported statistical results and associated study characteristics published in different languages have never been tested in environmental sciences. We address this knowledge gap by systematically searching published meta-analyses and comparing effect-size estimates between English- and Japanese-language studies included in existing meta-analyses. Of the 40 published ecological meta-analysis articles authored by those affiliated to Japanese institutions, we find that three meta-analysis articles searched for studies in the two languages and involved sufficient numbers of English- and Japanese-language studies, resulting in four eligible meta-analyses (i.e., four meta-analyses conducted in the three meta-analysis articles). In two of the four, effect sizes differ significantly between the English- and Japanese-language studies included in the meta-analyses, causing considerable changes in overall mean effect sizes and even their direction when Japanese-language studies are excluded. The observed differences in effect sizes are likely attributable to systematic differences in reported statistical results and associated study characteristics, particularly taxa and ecosystems, between English- and Japanese-language studies. Despite being based on a small sample size, our findings suggest that ignoring non-English-language studies may bias outcomes of ecological meta-analyses, due to systematic differences in study characteristics and effect-size estimates between English- and non-English languages. We provide a list of actions that meta-analysts could take in the future to reduce the risk of language bias.

La méta­analyse joue un rôle essentiel dans les synthèses de preuves quantitatives en écologie et en conservation de la biodiversité. La fiabilité des estimations dans les méta­analyses dépend fortement d'un échantillonnage non biaisé des études primaires. Bien que des études antérieures aient examiné les biais potentiels dans les méta­analyses écologiques, les biais n'ont jamais été mis à l'épreuve dans les sciences de l'environnement lors de publications dans différentes langues de résultats statistiques enregistrés et de caractéristiques des études associées. Nous abordons cette lacune dans les connaissances en recherchant de manière systématique des méta­analyses publiées et en comparant les estimations d'ampleur de l'effet entre des études en anglais et en japonais figurant dans les méta­analyses existantes. Sur 40 articles publiés de méta­analyse écologique dont les auteurs sont affiliés à des institutions japonaises, nous trouvons que trois articles de méta­analyse ont recherché des études dans les deux langues et ont mis en jeu des nombres suffisants d'études en anglais et en japonais, avec pour résultat quatre méta­analyses admissibles (c.­à­d. quatre méta­analyses effectuées dans les trois articles de méta­analyse). Dans deux de ces quatre cas, les ampleurs de l'effet diffèrent de façon significative entre les études en japonais et en anglais comprises dans les méta­analyses, ce qui provoque des changements considérables dans les ampleurs moyennes globales de l'effet et même dans leur direction lorsque les études en japonais sont exclues. On peut probablement attribuer les différences observées dans les ampleurs de l'effet entre les études en japonais et en anglais, en particulier en ce qui concerne les taxons et les écosystèmes, aux différences systématiques dans les résultats statistiques enregistrés, de même que dans les caractéristiques des études associées. Bien que basés sur un échantillon de petite taille, nos résultats suggèrent que le fait d'ignorer les études non anglophones peut biaiser les résultats de méta­analyses écologiques en raison des différences systématiques dans les caractéristiques d'étude et les estimations d'ampleur de l'effet entre langue anglaise et langues non anglophones. Nous proposons une liste de mesures qui pourraient être adoptées à l'avenir dans les méta­analyses pour réduire le risque de biais linguistique.

A metanálise desempenha um papel crucial na síntese de evidências quantitativas na ecologia e conservação da biodiversidade. A confiabilidade das estimativas nas metanálises depende fortemente da amostragem imparcial de estudos primários. Embora estudos anteriores tenham explorado possíveis vieses em metanálises ecológicas, os vieses nos resultados estatísticos relatados e características de estudos associados publicados em diferentes idiomas nunca foram testados em ciências ambientais. Abordamos essa lacuna de conhecimento pesquisando sistematicamente metanálises publicadas e comparando estimativas de tamanho de efeito entre os estudos em inglês e japonês incluídos nas metanálises existentes. Dos 40 artigos de metanálise ecológica publicados por autores filiados a instituições japonesas, descobrimos que três artigos de metanálise pesquisaram estudos nos dois idiomas e envolveram um número suficiente de estudos em inglês e japonês, resultando em quatro metanálises elegíveis (ou seja, quatro metanálises realizadas nos três artigos de metanálise). Em duas das quatro metanálises, os tamanhos de efeito diferem significativamente entre os estudos em inglês e japonês incluídos nas metanálises, causando mudanças consideráveis nos tamanhos de efeito médios em geral e até mesmo na sua direção quando os estudos em japonês são excluídos. As diferenças observadas nos tamanhos de efeito provavelmente são atribuíveis a diferenças sistemáticas nos resultados estatísticos relatados, bem como às características de estudos associados, particularmente táxons e ecossistemas, entre estudos em inglês e japonês. Embora baseados em um pequeno tamanho amostral, nossos resultados sugerem que ignorar estudos que não sejam em inglês pode influenciar os resultados de metanálises ecológicas, devido a diferenças sistemáticas nas características dos estudos e estimativas de tamanho de efeito entre o idioma inglês e o não­inglês. Fornecemos uma lista de medidas que metanalistas podem adotar no futuro para reduzir o risco de viés de idioma.

El meta­análisis juega un papel crucial en la síntesis de evidencia cuantitativa en ecología y conservación de la biodiversidad. La fiabilidad de las estimaciones en los meta­análisis depende en gran medida del muestreo imparcial de los estudios primarios. A pesar de que estudios previos han explorado posibles sesgos en meta­análisis ecológicos, sesgos en resultados estadísticos y características asociadas al estudio publicados en diferentes idiomas nunca han sido comprobados en ciencias ambientales. Abordamos esta brecha de conocimiento buscando sistemáticamente los meta­análisis publicados y comparando las estimaciones del tamaño del efecto entre los estudios en inglés y japonés incluidos en los meta­análisis existentes. De los 40 artículos de meta­análisis ecológicos publicados por aquellos afiliados a instituciones japonesas, encontramos que tres artículos de meta­análisis buscaron estudios en dos idiomas e involucraron un número suficiente de estudios en inglés y japonés, lo que resultó en cuatro meta­análisis elegibles (i.e., cuatro meta­análisis realizados en tres artículos de meta­análisis). En dos de los cuatro, los tamaños de los efectos difieren significativamente entre los estudios en inglés y japonés incluidos en los meta­análisis, lo que provoca cambios considerables en los tamaños de efectos medios generales e incluso su dirección cuando se excluyen los estudios en japonés. Las diferencias observadas en los tamaños de los efectos son probablemente atribuibles a las diferencias sistemáticas en los resultados estadísticos informados, así como a las características de los estudios asociados, particularmente los taxones y los ecosistemas, entre los estudios en inglés y japonés. A pesar de estar basados ​​en un tamaño de muestra pequeño, nuestros hallazgos sugieren que ignorar los estudios que no están en inglés puede sesgar los resultados de los meta­análisis ecológicos, debido a las diferencias sistemáticas en las características del estudio y a las estimaciones del tamaño del efecto entre el idioma inglés y el no inglés. Proporcionamos una lista de acciones que los meta­analistas podrían tomar en el futuro para reducir el riesgo de sesgo lingüístico.